CM 
< 

CO 
O) 

to 
10 

O 

CL 
LU 



(19) 



(12) 



Europaisches Patentamt 
European Patent Office 
Off ice europeen des brevets (11) EP 0 915 598 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

12.05.1999 Bulletin 1999/19 

(21 ) Application number: 981 18038.3 

(22) Date of filing: 23.09.1 998 



(51) Int. CI. 6 : H04L 29/06, H04L 12/56, 
H04L 12/18 



(84) Designated Contracting States: 


(72) Inventor: Bushmitch, Dennis 


AT BE CH CY DE DK ES R FR GB GR IE IT LI LU 


Summit, New Jersey 07901 (US) 


MCNLPT SE 




Designated Extension States: 


(74) Representative: 


AL LT LV MK RO SI 


Lindner, Michael, Dipl.-lng. et al 




Patentanwalte, 


(30) Priority: 30.10.1997 US 960799 


Witte, Weller & Partner, 


(71) Applicant: 


Rotebuhtstrasse 121 


70178 Stuttgart (DE) 


MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD. 


Osaka 571 (JP) 





rn 

CO 



(54) Distributed internet protocol-based real-time multimedia streaming architecture 



(57) Multiple media push engines communicate 
with the multimedia client through a multi casting net- 
work that may incorporate multiple delivery paths. The 
streaming data representing media selections for deliv- 
ery are distributed across multiple media push engines 
using a non-hierarchial coding technique in which the 
data are represented as a set of substream compo- 
nents, capable of being reconstituted from fewer than all 
of the components of the original data stream. The 
higher the number of components used in reconstitu- 
tion, the higher the quality of service is provided by the 



reconstituted stream. Admission control to the group 
multicast session is administered in a distributed fash- 
ion, where an admission control unit opens the multicast 
stream, with all subsequent admission control decisions 
being made by the media push engines themselves. 
Substream component data are sent using Real-Time 
transport protocol while session management and the 
distributed admission control process are administered 
under the Real-Time Control Protocol. 
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Description 

Background and Summ ary of the Invention 

[0001 ] The present invention relates generally to net- 
worked multimedia systems. More particularly, the 
invention relates to a media delivery system for deliver- 
ing media selections to one or more media clients over 
a multicasting network. 

[0002] With the explosive growth of the Internet, there 
is a growing interest in using the Internet and other 
Internet protocol-based networks to deliver multimedia 
selections, such as video and audio material. Interactive 
television, movies on demand, and other multimedia 
push technologies are among the more promising appli- 
cations. 

[0003] The Internet is a connectionless network offer- 
ing best effort delivery service. Packets of data are 
routed as datagrams that carry the address of the 
intended recipient. A specific connection between the 
sender and the recipient is not required, because all 
host nodes on the network include the inherent capabil- 
ity to route datagrams from node to node until delivery is 
effected. This datagram packet delivery scheme is con- 
structed as a best effort delivery system in which the 
delivery of datagram packets is not guaranteed. Data- 
gram packets may be sent via different routes in the 
effort to increase the likelihood of delivery. Thus, if one 
node on the network is experiencing congestion, subse- 
quent datagrams may be alternately routed to avoid the 
congested node. This means that data datagram pack- 
ets do not inherently have a guaranteed arrival time. 
Even packets corresponding to a single message may 
be received out of order. This fact significantly affects 
how certain multimedia data are delivered. 
[0004] In many cases, multimedia data require real- 
time delivery. In the case of audio or video data, the 
data stream representing a particular media selection 
needs to be delivered in the proper time sequence, to 
allow the user to play back the audio or video selection 
live" as it is being sent. Clearly, if the datagram packets 
are delivered out of order, due to taking different deliv- 
ery routes, then playback at the multimedia client (e.g., 
a user's interactive TV) will be jumbled. 
[0005] The Real-time Protocol (RTP) is a current de 
facto standard for delivering real-time content over the 
Internet (or other networks based on an IP protocol). 
The Real-time Protocol replaces the conventional trans- 
mission control protocol (TCP) with a framework that 
real-time applications can use directly for data trans- 
port. Currently, the RTP standard supports a first type of 
message, namely one for carrying the media content 
data or streaming data. Typically, a separate protocol, 
the Real-Time Control Protocol (RTCP) is used with 
RTP to pass control messages for session manage- 
ment, rate adaptation and the like. 
[0006] While the Real-time Protocol can be used to 
deliver multimedia streaming data over computer net- 



works, the existing architecture has not proven robust 
enough to provide high quality presentation using best 
effort network services such as those provided by the 
Internet. The present invention solves this problem by 

5 using a distributed media push architecture that is capa- 
ble of supplying streaming data redundantly from multi- 
ple sources and over multiple distribution paths. The 
media push engines have associated media storage 
units that store streaming data as non-hierarchial sets 

w of substream components. The components are capa- 
ble of being reconstituted into a reconstructed stream 
from fewer than all of the components, such that the 
higher the number of components used in ^constitu- 
tion, the higher the quality of the reconstructed stream. 

15 [0007] Conventional systems use a hierarchial coding 
scheme that treats some components as more impor- 
tant than others. Thus, conventional systems typically 
need to expend considerable resources to guarantee 
that the more important components are always deliv- 

20 ered. In contrast, the media delivery system of the 
invention uses a non-hierarchial coding scheme, multi- 
ple description coding (MDC) that treats all components 
as equals. Thus, no special resources need to be allo- 
cated to ensure that a given set of substream compo- 

25 nents is delivered. Naturally, the higher number of 
components delivered, the higher the quality achieved; 
on the other hand, unlike with conventional hierarchial 
coding, loss of any given single packet does not appre- 
ciably degrade the signal quality. 

30 [0008] The distributed media delivery system also 
employs a distributed admission control system. The 
media client contacts a single admission control unit to 
request a given media selection, but thereafter the 
admission control decisions are handled in distributed 

35 fashion by the media push engines themselves. The 
admission control unit communicates the request to a 
plurality of media push engines distributed across the 
network and those push engines individually determine 
whether they can participate in the multicasting session. 

40 Thus, the individual media push engines each evaluate 
local traffic congestion to determine whether it is capa- 
ble of supplying the requested data stream. The admis- 
sion control unit is thus not involved in directly 
determining which media push engines should be 

45 admitted to a multicast group session. The admission 
control unit simply assigns the multicast group session 
address and then allows the admission process to pro- 
ceed autonomously, in a distributed fashion. 
[0009] For a more complete understanding of the 

so invention, its objects and advantages, reference may be 
had to the following specification and to the accompany- 
ing drawings. 

Brief Description of the Drawings 

55 

[0010] 

Figure 1 is a network datagram illustrating a pre- 
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ferred embodiment of the invention; 
Figure 2 is a detailed network datagram showing 
how two different data streams (X and Y) are dis- 
tributed across the network using non-hierarchial 
multiple description coding; 5 
Figure 3 is a data flow diagram illustrating one 
embodiment of multiple descriptive coding (MDC); 
Figure 4 is a layer datagram illustrating the TCP/IP 
architecture and also showing how the RTP archi- 
tecture may be integrated into an IP-based system; w 
Figure 5 is a detailed format datagram illustrating 
the packet format according to the Real-time Proto- 
col (RTP); 

Figure 6 is a network datagram illustrating the call 
admission and session management according to is 
the invention; 

Figure 7 is a detailed network datagram showing 
information flow between a media push engine and 
a multimedia client over the multicasting IP net- 
work; 20 
Figure 8 is a network datagram illustrating the proc- 
ess of source component server redistribution; and 
Figure 9 is a protocol datagram showing how the 
Real-time Protocol (RTP) may be modified to 
increase its reliability. 25 

Detailed Description of the Preferred Embodiment 

[001 1 ] Referring to Figure 1 , an exemplary distributed 
networked multimedia system is illustrated at 10. A plu- 30 
ralrty of media push engines 12 are accessible through 
the multicasting network 14. The presently preferred 
embodiment is designed to work over a network 
employing the Internet protocol (IP); however, the prin- 
ciples of the invention may be readily extended to net- 35 
works using other protocols. The network 14 is also 
assessable by one or more multimedia clients 16, as 
illustrated. An admission control unit 18. accessible 
through network 14, performs certain admission control 
functions, primarily to initiate or open a multicast group 40 
session. The admission control unit includes a catalog 
services system 20. The catalog services system con- 
tains a database record indicating which multimedia 
selections are available for delivery by the plurality of 
media push engines. Although involved in opening a 45 
multicast group session, the admission control process 
is actually performed in a distributed fashion as will be 
more fully discussed below. 

[0012] The distributed media delivery system 
responds to delivery requests from a multimedia client so 
by opening a multicast group session among the media 
client and those multimedia push engines that have the 
requested media selection available for delivery. Typi- 
cally multiple media push engines will participate in 
simultaneously delivering streaming data correspond- ss 
ing to the requested selection. The multimedia client is 
a user host that performs the presentation function. It 
reconstructs a final stream from the various stream 



components delivered by the participating media push 
engines. Each media push engine has its own data stor- 
age for the stream component data and those data stor- 
age systems can be mediated by a suitable distributed 
file system that provides a mountable and transparent 
storage and retrieval function. 

[0013] An important aspect of the distributed media 
delivery system is the manner in which streaming data 
are stored on the media push engines. Unlike traditional 
systems that store multimedia data in a hierarchial fash- 
ion, the present invention uses a non-hierarchial coding 
scheme, referred to here as multiple description coding 
(MDC). The multiple description coding splits the video 
and/or audio stream into substreams called compo- 
nents. Each component can then be coded and trans- 
mitted over the network independently from all other 
components. The client software on a multimedia client 
1 6 can assemble a reconstructed stream from any sub- 
set of the components. Thus the reconstructed stream 
can be assembled from fewer than all of the compo- 
nents. The higher the number of components used in 
reconstruction, the higher the quality of the recon- 
structed stream. 

[0014] Using this non-hierarchial coding to deliver 
streaming data over the inherently unreliable network 
affords surprisingly robust media delivery, particularly 
when multiple push engines participate in the delivery. 
As will be more fully discussed, the media push engines 
control the multicast group session admission process 
themselves, in a distributed fashion, adding or subtract- 
ing media push engines to the group session as needed 
to maintain a high quality of services. Thus, when the 
multicasting network 14 exhibits low traffic congestion, 
only a few media push engines may be needed to sup- 
ply all of the components of the MDC-encoded stream. 
Even if some components are not delivered in a timely 
fashion, the multimedia client will nevertheless be able 
to reconstruct the stream for presentation (with some 
degree of degraded quality). If the network traffic con- 
gestion is high, the media push engines negotiate with 
one another to add additional media push engines. 
Because the admission control process is distributed, 
individual media push engines are able to assess their 
own local traffic congestion and will thus participate in 
the group session, or not, depending on local traffic con- 
ditions. 

[001 5] Figure 2 shows in greater detail how the multi- 
ple description coding works. In Figure 2, two multime- 
dia streams, designated X and Y are stored across a 
plurality of media push engines. These streams are bro- 
ken into substream components, designated by sub- 
scripts. X 1( X 2 , .... X n ; Yt , Y 2 , ... Y n . Note that the 
substream components stored across the plurality of 
media push engines are not necessarily the same for 
each engine. Thus media push engine 12a stores com- 
ponents X 1t Xg and Y v Similarly, media push engine 
12b stores components X 2 , X 7 and Y 2 . 
[0016] The multimedia clients reassemble the data 
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stream of interest by summing the proper substream 
components in the proper order. Thus multimedia client 
16a reconstructs stream X as illustrated, while multime- 
dia 16b constructs stream Y as illustrated. At the multi- 
media client it matters not that individual substream 
components arrive through different paths from different 
media push engines. 

[0017] The presently preferred multiple description 
coding scheme is constructed as illustrated in Figure 3. 
The original multimedia data stream (e.g. video and/or 
audio data) is decomposed into multiple subsignals and 
then each subsignal is independently compressed. As 
noted above, the decomposition is non-hierarchial, such 
that an acceptable signal can be recovered from any 
one subsignal, an incremental improvement can be 
realized by additional subsignals, and a perfect recon- 
struction is achieved when all subsignals are received 
exactly. Moreover, it is preferable to maximize the over- 
all compression gain so long as the above three criteria 
are met. 

[001 8] One way to decompose the original signal is to 
construct each subsignal as a reduced resolution repre- 
sentation of the original signal. This may be done, as 
illustrated, by submitting the original signal through a 
low pass filter followed by downsampling. The subsig- 
nals thus differ only in the sampling positions. If desired, 
such decomposition can be obtained by a filter bank 
that includes the pre-f ilter and its shifted versions. 
[0019] The pre-f ilter should be selected to suppress 
the aliasing components in the downsampled subsig- 
nals. This helps to reduce the bit rates required for cod- 
ing the subsignals and to enable an acceptable image 
recovery from a single subsignal. 
[0020] The pre-f ilter should further be selected so that 
it does not completely eliminate the high frequency 
components. Were the high frequency components 
entirely eliminated, there would be no way to recover 
those components in the original signal, even when all 
subsignals are present. Thus the filter should suppress, 
but not entirely eliminate, the high frequency compo- 
nents. 

[0021] Mathematically, reconstruction of the sub- 
stream components into a reconstructed stream 
involves inverting the matrix equation relating the sam- 
ples in all the substreams and the samples in the origi- 
nal stream. Typically this may involve a large matrix 
equation, with inversion employing a large amount of 
computation and memory space. One way to address 
this computational burden is to use a block recursive 
reconstruction method. At each step in the recursive 
process a block of 2 x 2 samples in the original stream 
is recovered based on up to four corresponding sam- 
ples in the received subcomponent streams. Of course, 
other computational techniques may be employed to 
accomplish the same result. For more information on 
coding and encoding in a non-hierarchial fashion using 
multiple description coding, see "Robust Image Coding 
and Transport in Wireless Networks Using a Non-Hier- 



archial Decomposition," Yao Wang and Doo-man 
Chung. Mobile Multimedia Communications; Goodman 
(Plenum Press) 

[0022] The M DC -coded substreams are delivered 

s over the multicasting network as datagrams using the 
Real-Time Protocol (RTP) for content delivery and the 
Real-Time Protocol (RTCP) to implement flow control. 
These protocols are able to naturally coexist with the 
popular TCP/IP protocol used by many Internet applica- 

10 tions. Figure 4 presents a review of these protocols 
through an example of how five entities might communi- 
cate with one another. In Figure 5 entities 30-38 com- 
municate with each other using the layered architecture 
popularized by the Internet. It will, of course, be under- 

15 stood that Figure 4 is intended only to show how the 
presently preferred Real-Time Protocol (RTP) inte- 
grates in one possible architectural scheme. Although 
the Real-Time Protocol is presently preferred, this is not 
intended as a limitation of the invention in its broader 

20 aspects. On the contrary, other message delivery proto- 
cols may be suitably used. 

[0023] In Figure 4 each of the communicating entities 
30-38 has been illustrated as a layered architecture, 
with the physical layer at the bottom and the application 

25 layer at the top. Entity 30 communicates with entity 32 
using the Ethernet Protocol at the physical level. Entity 
32 and entity 34 communicate at the physical level 
using the ATM Protocol. In a like fashion, entity 34 com- 
municates with entity 36 using the Ethernet Protocol, 

30 and entity 36 communicates with entity 38 using the 
PPP Protocol. Again, the physical layer communication 
protocols selected for illustration here are not intended 
as a limitation upon the invention as set forth in the 
appended claims. 

35 [0024] Above the physical layer is the Internet Proto- 
col (IP) layer. The IP Protocol isolates the physical or 
transport layer from the application layer. The IP Proto- 
col supports connectionless communication in which 
packets of information are sent and received as data- 

40 grams. Note that in the illustrated example of Figure 4 
all communicating entities are using the IP Protocol. 
[0025] One layer above the IP Protocol two different 
higher level protocols have been illustrated, the TCP 
Protocol and the UDP Protocol. Again, the illustrations 

45 are intended only as one example of a possible configu- 
ration. The UDP Protocol or User Datagram Protocol 
represents a simple transport protocol. It makes no 
attempt to preserve the sequence of messages it deliv- 
ers. The TCP Protocol or Transmission Control Protocol 

so provides a higher level of reliability, yet insures that dat- 
agrams are delivered in the proper sequence. The TCP 
Protocol uses an acknowledgment system to insure that 
all datagrams are delivered in the proper sequence. The 
TCP Protocol includes a mechanism for retransmitting 

55 packets that have not been acknowledged. This 
acknowledgment/retransmit technique guarantees 
proper packet delivery, but it does not guarantee packet 
delivery in real-time. Thus TCP Protocol is usually 
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unsuitable for delivering real-time data such as multime- 
dia video and/or audio data. 

[0026] The Real-Time Protocol (RTP) replaces the 
more sophisticated Transport Protocol of TCP with a 
simple framework that applications can use directly. 
Instead of implementing a missing data detection and 
retransmission mechanism-which can introduce trans- 
mission delays-the RTP Protocol simply ignores miss- 
ing data. The RTP Protocol is also not typically 
concerned with the sequence of packet delivery. The 
protocol assumes that the application layer above it will 
correct any misordered data. The RTP Protocol is com- 
patible with a number of different encoding standards, 
such as MPEG, JPEG and H.261 . 
[0027] In the illustrated example of Figure 4, entities 
30 and 38 are both running RTP Protocol. Thus stream- 
ing data could be supplied from entity 30 to entity 38 
through the network consisting of entities 30, 32, 34, 36 
and 38. 

[0028] The RTP Protocol is designed for multicast 
operation. Multicasting is a form of message broadcast- 
ing in which messages may be delivered to many differ- 
ent recipients in a designated set. Multicast addresses 
identify sets of interfaces, frequently including multiple 
interfaces belonging to different systems. When a mes- 
sage has a multicast destination address, the network 
strives to deliver it to all interfaces in the set. This func- 
tion lets a system generate a message once and have 
that message delivered to many different recipients. 
[0029] Aside from delivering the datagram packets to 
multiple recipients, a multicasting network will typically 
also support feedback from the message recipients. 
Typically all participants in the multicasting group ses- 
sion can receive these feedback messages. Such feed- 
back messages are commonly used for real-time traffic 
control, following a related Real-Time Control Protocol 
(RTCP). In some respects, RTCP is an optional exten- 
sion to RTP RTCP packets are used by the presently 
preferred embodiment to send flow control and session 
management information among the entities participat- 
ing in the group multicast session. 
[0030] Figure 5 illustrates the RTP packet format. 
Note that the packet includes a sequence number and 
time stamp used in reassembling the packets in the 
proper time sequence. 

[0031] Figures 6 and 7 show the details of how the 
multimedia client, admission control unit and media 
push engines communicate with one another during a 
multicast group session. Specifically. Figure 6 shows 
the basic message flow and communication sequence 
of the preferred embodiment. Figure 7 gives a more 
detailed view of how the RTP and RTCP Protocols are 
used in routing the substream component datagrams 
from media push engine to multimedia client. 
[0032] Referring first to Figure 6, the multimedia client 
16 sends a unicast TCP Protocol message to the 
Admission Control Unit 18, requesting that a particular 
media selection commence delivery. The Admission 



Control Unit 18 consults its catalog services system 20 
to determine if the requested selection (i.e. requested 
stream) is present on the network Assuming the stream 
is present, the Admission Control Unit transmits a 

5 Stream Open message to those Media Push Engines 
12 that have at least some substream components of 
the requested selection. This Open Stream request is 
sent to all Media Push Engines. Those Media Push 
Engines that find themselves capable of serving the 

w requested stream components jointly enter the multi- 
casting session management and flow control session 
between only those hosts serving and receiving that 
specific stream. Participating Media Push Engines and 
participating multimedia clients obtain the needed multi- 

is cast address for such control multicast group session 
from the Admission Control Unit. Thereafter, the Admis- 
sion Control Unit effectively drops out of the session, 
allowing subsequent session management and flow 
control messages to be exchanged only among multi- 

20 cast group members (that is, the multimedia client and 
all corresponding media push engines). This reduces 
the overhead on the Admission Control Unit 18. 
[0033] The Admission Control Unit generates a multi- 
cast Class D address for use by the multicast session. 

25 This address may be selected from a pool of available 
multicast address entries. The Admission Control Unit is 
thus responsible for managing the allocation of multi- 
cast addresses. When a multicast session is ended the 
Admission Control Unit returns the multicast session 

30 address back to the pool of available addresses. 

[0034] Thus once the multicast group session is initi- 
ated, those media push engines able to supply sub- 
stream components will do so by sending unicast RTP 
session stream data to the multimedia client 16. 

35 Through the RTCP Protocol these media push engines 
may communicate with one another to join or depart a 
multicast group session, as needed to maintain a high 
quality of service. 

[0035] Referring to Figure 7, the media push engine 

40 and multimedia client communicate through the network 
at two different levels. As illustrated by the dotted lines, 
a unicast RTP session transmits the multimedia stream- 
ing data to the multimedia client. Concurrently, as 
required, the media push engine and multimedia client 

45 send each other RTCP reports, specifically sender 
reports and receiver reports as well as any appropriate 
flow control commands and other session management 
commands (e.g. Start Push, Pause, Continue). The 
RTCP control signals are shown by the bi-directional 

so solid line in Figure 7. 

[0036] Essentially, after the Stream Open message is 
sent by the Admission Control Unit, each mediapush 
engine consults its associated media storage system to 
determine whether it is capable of serving the 

55 requested stream components. If so, then the media 
push engine joins the specified multicast group. Other- 
wise it does not participate in the multicast session 
(unless later requested). Once the media push engine 
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has joined the multicast group, it participates in commu- 
nications using the RTCP Protocol whereby statistics of 
sent data and received data are exchanged among the 
members of the group. As noted above, the Admission 
Control Unit does not need to participate in these com- 
munications and hence it will remain dormant unless a 
request for another session is made or until the current 
session is requested to be terminated. 
[0037] Effectively, the system implements a distributed 
Admission Control System, where the participating 
members of the group collectively and distributively 
make the admission control decisions. One benefit of 
this distributed approach is that the invention can incor- 
porate intelligent mechanisms to prevent network con- 
gestion and to improve quality of service, despite the 
fact that the multicasting network is a best effort network 
with no guaranteed real-time delivery 
[0038] Best effort networks, particularly those lacking 
sophisticated traffic and user control policies, experi- 
ence frequent congestions. Such congestions can 
result in a loss of or substantial delay of real-time data. 
As previously discussed, real-time data that is delivered 
late is effectively treated as not delivered. The continu- 
ous influx of data into a congested node of the network 
tends to make the congestion even worse. The present 
invention uses RTCP sender reports and received 
reports on time-sensitive transmissions as an indication 
that a given node may need to scale back on the 
number of components it is transmitting when conges- 
tion is detected. 

[0039] Figure 4 illustrates how this may be accom- 
plished. The multimedia client 16 has requested the X 
real-time data stream, consisting of sub-stream compo- 
nents Xl X 2 . X 3 and X 4 . Assume that Media Push 
Engine 12 in Figure 7 is experiencing local traffic con- 
gestion such that sub-stream components are arriving 
late at the multimedia client 16. The multimedia client's 
RTCP receiver report notifies the Media Push Engine 
12 (and ail other media push engines participating in the 
group session) that some percentage of the component 
data from Media Push Engine 1 2. Media Push Engine 
1 2 analyzes these reports and stops sending a selected 
component, in this case the X 3 , thereby decreasing the 
amount of traffic flowing through its point of congestion. 
Thus, after adjustment, Media Push Engine 12 supplies 
only components X 1f X 2 and X4 to the multimedia client. 
As the other media push engines participating in the 
group session receive the same sender and receiver 
reports, the loss of the X 3 component from Media Push 
Engine 12 may be compensated for if other push 
engines are able to supply this missing component. 
Otherwise, the quality of service will be slightly 
degraded as discussed above. 
[0040] Figure 8 illustrates how the data stream may be 
effectively redistributed by making local adjustments to 
the substream components being sent. In the illustrated 
example assume that there is local congestion some- 
where in the data path that supplies substream compo- 



nents from Media Push Engine 12b. The RTCP sender 
and receiver reports will thus indicate that some portion 
of the components previously sent to the multimedia cli- 
ent 16 by Media Push Engine 12b are lost or delayed 

5 due to local congestion. In the illustrated example the 
lost components happen also to be present in the stor- 
age system of Media Push Engine 12a. Media Push 
Engine 12a can either retransmit the lost component 
payloads to the multimedia client or simply adjust the 

10 set of components to be transmitted in future real-time 
data transactions. In the case where lost payloads are 
retransmitted by another media push engine, sufficient 
buffering should be supplied at the multimedia client to 
allow the missing components to be reassembled with 

15 the previously delivered components before the stream 
is reconstructed and presented to the user. In the case 
where the system merely alters the component set for 
future transmissions, such change constitutes a scala- 
ble server component redistribution mechanism. This 

20 mechanism promotes improved quality of service by 
improving the presentation of stream data to the multi- 
media client. 

[0041] Although the embodiment described above is 
generally suitable for most media delivery applications, 

25 there are some systems that cannot tolerate even a 
slight degradation in quality. Such systems include high 
quality broadcast video distribution. In these more 
demanding applications, the system of the previously 
described preferred embodiment can be modified to 

30 employ an additional reliability mechanism for the real- 
time component. In this case the Real-Time Protocol 
may be modified or augmented to allow retransmissions 
of lost real-time payloads. This "Reliable RTP" is illus- 
trated in Figure 9. The Media Push Engine communi- 

35 cates to the RTP stack using Real-Time Protocol. In this 
case assume that the first and third components are 
received but the second component is missing. There is 
an immediate negative acknowledgment (NACK) from 
the RTP stack, telling the Media Push Engine that the 

40 second payload was not received. The Media Push 
Engine then retransmits the needed payload and the 
RTP stack places the needed payload into the correct 
location within the data buffer. The client application 
then reads the data from the data buffer. Any duplicate 

45 packets are dropped and any excessively delayed pack- 
ets may also be dropped. 

[0042] From the foregoing it will be understood that 
the invention provides a media delivery system architec- 
ture that employs a distributed, networked technique for 

so delivering streaming data over a best effort network. 
The architecture may be readily scaled, larger or 
smaller, as the server's complexity increases linearly 
with the number of clients. The architecture is thus a 
fully distributed, tightly coupled parallel architecture that 

55 is capable of providing simple and yet robust service. 
[0043] Through the use of multiple description coding 
(MDC) and multiple path transport, the invention can 
provide a high quality of service without resorting to 
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delay-producing transmission retry techniques. Thus 
the invention will readily work with existing Real-Time 
Transport Protocol (RTP) for data transport and Real- 
Time Control Protocol (RTCP) for session manage- 
ment, rate adaptation and the like. When congestion is 5 
encountered the presentation can be downscaled with- 
out interruption, thanks to the multiple description cod- 
ing and the manner in which participants of a group 
session can be added to or removed from the group. 
The flow control of streams is also controllable through w 
these same mechanisms to prevent or reduce network 
congestion once it is detected. 

[0044] The invention is therefore ideally suited for 
delivery of multimedia selections as well as video and 
audio streaming data. The invention will readily support is 
data streams of multiple bit rates and it is capable of 
providing services at both constant bit rates and varia- 
ble bit rates. 

[0045] While the invention has been described in its 
presently preferred embodiments, it will be understood 20 
that the invention is capable of certain modification and 
change without departing from the spirit of the invention 
as set forth in the appended claims. 

Claims 25 

1. A distributed media delivery system for delivering 
media selections to a media client (16) over a mul- 
ticasting network (14), characterized by: 

30 

a plurality of media push engines (12) accessi- 
ble through said network (14), said push 
engines (12) each having associated media 
storage unit or storing streaming data repre- 
senting the media selections available for deliv- 35 
ery; 

said media storage units being configured to 
store said streaming data as a non-hierarchical 
set of substream components capable of being 
reconstituted into a reconstructed stream from 40 
fewer than all of said components, such that 
the higher the number of components used in 
reconstitution, the higher quality the recon- 
structed stream; and 

an admission control system (18, 20) accessi- 45 
ble through said network (14), said admission 
control system (18, 20) including a catalog for 
storing the identity of the media selections 
available for delivery by each of said media 
push engines (12), 50 
said admission control system (18, 20) being 
operative, in response to a request for a given 
media selection from a media client (16), to 
open a multicast group session among said 
media client (16) and at least a portion of said 55 
media push engines (12) having the given 
media selection available for delivery, 
whereby said media push engines participating 



in said multicast group session each supply to 
said network those substream components 
corresponding to the given media selection, for 
delivery to and reconstitution by said media cli- 
ent. 

2. "The media delivery system of claim 1, character- 
ized in that said admission control system (18, 20) 
is a distributed system defined at least in part 
through interaction between said media push 
engines (12). 

3. The media delivery system of claim 1, character- 
ized in that said media push engines (12) communi- 
cate with said network (14) over different 
communication paths. 

4. The media delivery system of claim 1, character- 
ized in that said network (14) is a connectionless 
network offering best effort delivery services. 

5. The media delivery system of claim 1, character- 
ized in that said network (14) is the Internet. 

6. The media delivery system of claim 1 , character- 
ized in that said media client (16) and said media 
push engines (12) participating in said multicast 
group session employ Real-Time Transport Proto- 
col (RTP) for data transport. 

7. The media delivery system of claim 1, character- 
ized in that said media client (16) and said media 
push engines (12) participating in said multicast 
group session employ Real-Time Control Protocol 
(RTCP) for session management. 

8. The media delivery system of claim 1, character- 
ized in that at least a portion of said substream 
components are replicated across several media 
push engines (12). 

9. The media delivery system of claim 1, character- 
ized in that at least a portion of said substream 
components are replicated across first and second 
media push engines (1 2) and that said delivery sys- 
tem further comprises congestion handling system 
for identifying one of said first and second media 
push engines (12) as the cause of congestion and 
for automatically invoking the other of said first and 
second media push engines (12) to participate in 
said multicast group session. 

10. The media delivery system of claim 9, character- 
ized by data buffering system associated with said 
media client (16) for storing substream components 
prior to reconstitution. 

11. The media delivery system of claim 1, character- 
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ized in that said admission control unit (18, 20) is 
further operative, in response to a request from said 
media client (16) to end a multicast group session, 
to instruct all media push engines (12) participating 
in said multicast group session to terminate the 5 
session. 

12. The media delivery system of claim 1. character- 
ized in that said admission control system (18, 20) 
includes an admission control unit (18) that main- 10 
tains a pool of multicast session addresses for use 

in invoking multicast group sessions and wherein 
said admission control unit (18) assigns a desig- 
nated multicast session address, selected from 
said pool, for use by said multicast group session. 15 

13. The media delivery system of claim 12, character- 
ized in that said admission control unit (18) is fur- 
ther operative, in response to a request from said 
media client (16) to end a multicast group session, 20 
to return said designated multicast session address 

to said pool. 

14. The media delivery system of claim 12, character- 
ized in that said media client (16) and said media 25 
push engines (12) participate in said multicast 
group session exchange flow control messages 
without involving said admission control unit (18). 

15. The media delivery system of claim 1, character- 30 
ized in that between said media client (16) and 
every media push engine (12) participating in said 
multicast group session there is a unicast flow of 
datagrams containing real-time stream component 
data. 35 
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